Modelling and tracking objects with a topology preserving self-organising neural network

Human gestures form an integral part in our everyday communication. We use gestures not only to reinforce meaning, but also to describe the shape of objects, to play games, and to communicate in noisy environments. Vision systems that exploit gestures are often limited by inaccuracies inherent in handcrafted models. These models are generated from a collection of training examples which requires segmentation and alignment. Segmentation in gesture recognition typically involves manual intervention, a time consuming process that is feasible only for a limited set of gestures. Ideally gesture models should be automatically acquired via a learning scheme that enables the acquisition of detailed behavioural knowledge only from topological and temporal observation. The research described in this thesis is motivated by a desire to provide a framework for the unsupervised acquisition and tracking of gesture models. In any learning framework, the initialisation of the shapes is very crucial. Hence, it would be beneficial to have a robust model not prone to noise that can automatically correspond the set of shapes. In the first part of this thesis, we develop a framework for building statistical 2D shape models by extracting, labelling and corresponding landmark points using only topological relations derived from competitive hebbian learning. The method is based on the assumption that correspondences can be addressed as an unsupervised classification problem where landmark points are the cluster centres (nodes) in a high-dimensional vector space. The approach is novel in that the network can be used in cases where the topological structure of the input pattern is not known a priori thus no topology of fixed dimensionality is imposed onto the network. In the second part, we propose an approach to minimise the user intervention in the adaptation process, which requires to specify a priori the number of nodes needed to represent an object, by utilising an automatic criterion for maximum node growth. Furthermore, this model is used to represent motion in image sequences by initialising a suitable segmentation that separates the object of interest from the background. The segmentation system takes into consideration some illumination tolerance, images as inputs from ordinary cameras and webcams, some low to medium cluttered background avoiding extremely cluttered backgrounds, and that the objects are at close range from the camera. In the final part, we extend the framework for the automatic modelling and unsupervised tracking of 2D hand gestures in a sequence of k frames. The aim is to use the tracked frames as training examples in order to build the model and maintain correspondences. To do that we add an active step to the Growing Neural Gas (GNG) network, which we call Active Growing Neural Gas (A-GNG) that takes into consideration not only the geometrical position of the nodes, but also the underlined local feature structure of the image, and the distance vector between successive images. The quality of our model is measured through the calculation of the topographic product. The topographic product is our topology preserving measure which quantifies the neighbourhood preservation. In our system we have applied specific restrictions in the velocity and the appearance of the gestures to simplify the difficulty of the motion analysis in the gesture representation. The proposed framework has been validated on applications related to sign language. The work has great potential in Virtual Reality (VR) applications where the learning and the representation of gestures becomes natural without the need of expensive wear cable sensors

Generic colour image segmentation via multi-stage region merging

We present a non-parametric unsupervised colour image segmentation system that is fast and retains significant perceptual correspondence with the input data. The method uses a region merging approach based on statistics of growing local structures. A two-stage algorithm is employed during which neighbouring regions of homogeneity are traced using feature gradients between groups of pixels, thus giving priority to topological relations. The system finds spatially cohesive and globally salient image regions usually without losing smaller localised areas of high saliency. Unoptimised implementations of the method work nearly in real-time, handling multiple frames a second. The system is successfully applied to problems such as object detection and tracking

Automatic landmark extraction from a class of hands using growing neural gas

A new method for automatically building statistical shape models from a set of training examples and in particular from a class of hands. In this method, landmark extraction is achieved using a self-organising neural network, the Growing Neural Gas (GNG), which is used to preserve the topology of any input space. Using GNG, the topological relations of a given set of deformable shapes can be learned. We describe how shape models can be built automatically by posing the correspondence problem on the behaviour of self-organising networks that are capable of adapting their topology to an input manifold, and due to their dynamic character to readapt it to the shape of the objects. Results are given for the training set of hand outlines, showing that the proposed method preserves accurate models

Measuring the Engagement of the Learner in a Controlled Environment Using Three Different Biosensors

Irrespective of the educational model, the major challenge is how to achieve maximum efficiency of the education process and keep learners engaged during learning. This paper investigates the relationship between emotions and engagement in the E-learning environment, and how recognizing the learners emotions and changing the content delivery accordingly can affect the efficiency of the E-learning process. The proposed experiment aims to identify ways to increase the engagement of the learners, hence, enhance the efficiency of the learning process and the quality of learning. A controlled experiment was conducted to investigate participants emotions using bio sensors such as eye tracker, EEG, and camera to capture facial images in different emotional states. One-way analysis of variance (ANOVA) test and t-Test was carried out to compare the performance of the three groups and show if there was an effect of using the Affective E-learning system to improve the learners performance. Our findings support the conclusion that using bio sensors as a quantitative research tool to investigate human behaviours and measure emotions in real time can significantly enhance the efficiency of E-learning

Affective Computing to Enhance E-Learning in Segregated Societies

According to UN Women, to build stronger economies, it is essential to empower women to participate fully in economic life across all sectors. Increasing women and girls’ education enhances their chances to participate in the labor market. In certain cultures, like in Saudi Arabia, women contribution to the public economy growth is very limited. According to the World Bank, less than 20 percent of the female population participate in the labor force. This low participation rate has many reasons. One of them, is the educational level and educational quality for females. Although Saudi Arabia has about thirty three universities, opportunities are still limited for women because of the restrictions of access put upon them. A mixture of local norms, traditions, social beliefs, and principles preventing women from receiving full benefits from the educational system. Gender segregation is one of the challenges that limits the women access for education. It causes a problem due to the shortage of female faculty throughout the country. To overcome this problem, male faculty are allowed to teach female students under certain regulations and following a certain method of education delivery and interaction. However, most of these methods lack face-to-face communication between the teacher and students, which lowers the interactivity level and, accordingly, the students’ engagement, and increases the need for other alternatives. The e-learning model is one of high benefit for female students in such societies. Recognizing the students’ engagement is not straightforward in the e-learning model. To measure the level of engagement, the learner’s mood or emotions should be taken into consideration to help understanding and judging the level of engagement. This paper is to investigate the relationship between emotions and engagement in the e-learning environment, and how recognizing the learner’s emotions and change the content delivery accordingly can affect the efficiency of the e-learning process. The proposed experiment alluded to herein should help to find ways to increase the engagement of the learners, hence, enhance the efficiency of the learning process and the quality of learning, which will increase the chances and opportunities for women in such societies to participate more effectively in the labor market

Exploring the Suitability of Semantic Spaces as Word Association Models for the Extraction of Semantic Relationships

Given the recent advances and progress in Natural Language Processing (NLP), extraction of semantic relationships has been at the top of the research agenda in the last few years. This work has been mainly motivated by the fact that building knowledge graphs (KG) and bases (KB), as a key ingredient of intelligent applications, is a never-ending challenge, since new knowledge needs to be harvested while old knowledge needs to be revised. Currently, approaches towards relation extraction from text are dominated by neural models practicing some sort of distant (weak) supervision in machine learning from large corpora, with or without consulting external knowledge sources. In this paper, we empirically study and explore the potential of a novel idea of using classical semantic spaces and models, e.g., Word Embedding, generated for extracting word association, in conjunction with relation extraction approaches. The goal is to use these word association models to reinforce current relation extraction approaches. We believe that this is a first attempt of this kind and the results of the study should shed some light on the extent to which these word association models can be used as well as the most promising types of relationships to be considered for extraction

Assessing the Effectiveness of Automated Emotion Recognition in Adults and Children for Clinical Investigation

Recent success stories in automated object or face recognition, partly fuelled by deep learning artificial neural network (ANN) architectures, has led to the advancement of biometric research platforms and, to some extent, the resurrection of Artificial Intelligence (AI). In line with this general trend, inter-disciplinary approaches have taken place to automate the recognition of emotions in adults or children for the benefit of various applications such as identification of children emotions prior to a clinical investigation. Within this context, it turns out that automating emotion recognition is far from being straight forward with several challenges arising for both science(e.g., methodology underpinned by psychology) and technology (e.g., iMotions biometric research platform). In this paper, we present a methodology, experiment and interesting findings, which raise the following research questions for the recognition of emotions and attention in humans: a) adequacy of well-established techniques such as the International Affective Picture System (IAPS), b) adequacy of state-of-the-art biometric research platforms, c) the extent to which emotional responses may be different among children or adults. Our findings and first attempts to answer some of these research questions, are all based on a mixed sample of adults and children, who took part in the experiment resulting into a statistical analysis of numerous variables. These are related with, both automatically and interactively, captured responses of participants to a sample of IAPS pictures

Robust modelling and tracking of NonRigid objects using Active-GNG

This paper presents a robust approach to nonrigid modelling and tracking. The contour of the object is described by an active growing neural gas (A-GNG) network which allows the model to re-deform locally. The approach is novel in that the nodes of the network are described by their geometrical position, the underlying local feature structure of the image, and the distance vector between the modal image and any successive images. A second contribution is the correspondence of the nodes which is measured through the calculation of the topographic product, a topology preserving objective function which quantifies the neighbourhood preservation before and after the mapping. As a result, we can achieve the automatic modelling and tracking of objects without using any annotated training sets. Experimental results have shown the superiority of our proposed method over the original growing neural gas (GNG) network

Performance Evaluation of a Statistical and a Neural Network Model for Nonrigid Shape-Based Registration

Shape-based registration methods frequently encounters in the domains of computer vision, image processing and medical imaging. The registration problem is to find an optimal transformation/mapping between sets of rigid or nonrigid objects and to automatically solve for correspondences. In this paper we present a comparison of two different probabilistic methods, the entropy and the growing neural gas network (GNG), as general feature-based registration algorithms. Using entropy shape modelling is performed by connecting the point sets with the highest probability of curvature information, while with GNG the points sets are connected using nearest-neighbour relationships derived from competitive hebbian learning. In order to compare performances we use different levels of shape deformation starting with a simple shape 2D MRI brain ventricles and moving to more complicated shapes like hands. Results both quantitatively and qualitatively are given for both sets